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Executive Summary 


Social mobility and improving the performance of disadvantaged pupils is a focus for policy 
in England, Northern Ireland and Wales. The findings from PISA 2018 provide a unique 
opportunity to explore the impact of disadvantage on pupils aged 15, to contextualise 
achievement outcomes with information about home-life and school, as well as providing 
an international perspective. 


How well are England, Northern Ireland and Wales doing at 
supporting the reading, maths and science performance of 
disadvantaged pupils compared with other countries, and has 
this changed over time? 


In all participating countries, more advantaged pupils performed better than their 
disadvantaged peers in reading, maths and science. Wales had more equitable outcomes, 
on average, than in England; the gap in performance between most- and least- 
disadvantaged pupils was statistically significantly greater in England than in Wales for 
reading, maths and science. Northern Ireland sits somewhere in the middle, as the gap 
between most- and least- disadvantaged pupils was not significantly different from either 
England or Wales. 


It is possible to be high achieving and have comparatively good equity in education. 
Countries previously identified as having greater equity in their education include Macao 
(China), Estonia and Canada; three countries which also outperformed England, Northern 
Ireland and Wales in all three PISA domains in 2018. Estonia and Canada, therefore, have 
been chosen as the comparator countries for this research alongside the Republic of 
Ireland. 


In England, Northern Ireland and Wales, socio-economic background had a smaller impact 
on performance amongst some of the most disadvantaged pupils than for their more 
advantaged peers. That is, there was a smaller difference in average achievement 
between a disadvantaged pupil and a marginally less disadvantaged peer but a larger 
difference in achievement between an advantaged pupil and a marginally less advantaged 
peer. In England, pupils who were more advantaged performed similarly to their peers in 
Canada and Estonia, indicating the importance of successfully improving the performance 
of disadvantaged pupils. 


Reading was assessed in greater depth in PISA 2018. The gap between disadvantaged 
pupils and their non-disadvantaged peers was similar for all three reading processes: 
locating information; understanding; and evaluating and reflecting. This indicates there 
was no process in which disadvantaged pupils were disproportionately weak. 


Encouragingly, in England, Northern Ireland and Wales, the 2018 average scores for 
disadvantaged pupils in maths and reading were significantly higher than in previous 
cycles. Disadvantaged pupils’ performance in science in each of the three countries has 
remained stable over time, whilst it has decreased in the comparator countries. 


How do resilient pupils’ (high-achieving, disadvantaged 
pupils’) attitudes differ from similarly-performing pupils’ from 
more advantaged backgrounds? 


Around a third of the disadvantaged pupils in England, Northern Ireland and Wales were 
defined as ‘resilient’. Resilient pupils are amongst the 33% most socio-economically 
disadvantaged pupils in their country who achieve at or above “level 3” in PISA reading, 
maths and science; or, more simply, they are disadvantage pupils who, despite the odds, 
have attainment in PISA of a level considered to equip them for success in later life. 


For the most part, there were no attitudinal differences found between resilient pupils and 
their similarly-achieving, more affluent peers. For example, no differences were found 
between pupils’ sense of belonging at school, life satisfaction or future aspirations. The 
exception to this was in Northern Ireland, where resilient pupils were less confident in their 
reading than their more affluent peers. 


It was mainly indicators of family poverty which distinguished these pupils. Whilst it is 
perhaps unsurprising that wealth indicators demonstrate the differences between resilient 
pupils and those who are not disadvantaged and similarly achieving, it does emphasise 
that households need sufficient household income to enable children to widen their 
horizons. 


In what circumstances do disadvantaged pupils tend to 
overcome barriers to perform better and how do their attitudes 
differ from low-performing disadvantaged pupils’ from similar 
backgrounds? 


Resilient pupils tended to use metacognitive strategies, had a growth mind-set and had 
high aspirations for their future education or careers. They were also less likely to truant. 
Resilient pupils were less likely to report having found meaning in life and less likely to 
report regularly feeling positive emotions; these unexpected findings would benefit from 
additional research to understand further. 


Are there lessons that can be applied more widely? 


Drawing on our findings around resilient pupils we recommend: 


e programmes that support metacognition could be beneficial for disadvantaged 
pupils 

e work towards countering the belief that intelligence is fixed for disadvantaged 
pupils 

e further investigation into the connection between aspiration and resilience to 


identify how they are related and, therefore, how and where more targeted 
support would be most beneficial for disadvantaged pupils. 


1 Introduction 


Differences in achievement between disadvantaged pupils and their more advantaged 
peers are a focus for policy in all parts of the UK. A gap in educational outcomes between 
pupils from less-affluent backgrounds compared to their peers is evident by the time they 
start school and only widens as they move through the school years (EEF, 2017). 
Disadvantaged pupils are more likely to leave school without GCSEs in English and maths 
than their peers (Children’s Commissioner 2019; EEF, 2017; Burn, 2015) and are 
underrepresented in professional occupations (Social Mobility Commission, no date). This 
means it is vital to understand ways in which we can support disadvantaged pupils, and 
ensure that they are given the tools they need to overcome barriers and succeed in later 
life. Whilst this is always an essential task, it has become even more important at a time 
when COVID-19 appears to have increased the negative impact of disadvantage on 
educational success (OECD, 2020). 


In 2018, nearly 80 countries participated in the Programme for International Student 
Assessment (PISA). PISA assesses 15 year olds’ proficiency in reading, maths and 
science and collects data on a wealth of background factors via questionnaires from 
pupils, teachers and school leaders. This provides a unique opportunity to explore the 
impact of disadvantage on pupils from an international perspective, as well as a 
comparison across England, Northern Ireland and Wales’ on the devolved matter of 
education. 


This report focuses on reading, mathematics and science performance in PISA to answer 
three questions: 


1, How well are England, Wales and Northern Ireland doing at supporting the reading, 
maths and science performance of disadvantaged pupils compared with other 
countries, and has this changed over time? 


2. How do resilient pupils’ (high-achieving, disadvantaged pupils’) attitudes differ from 
low-performing pupils’ from similar backgrounds and similarly performing pupils’ 
from more advantaged backgrounds? 


oO: In what circumstances do disadvantaged pupils tend to overcome barriers to 
perform better, and are there lessons that can be applied more widely? 


1 This research was commissioned as part of the PISA 2018 national centre for England, Wales and 
Northern Ireland. Analysis of Scotland was out of scope. 


2 Background 


Social mobility and improving performance of disadvantaged pupils is a 
priority for all parts of the UK 


In England, Wales and Northern Ireland, ‘disadvantaged’ is primarily defined and 
measured as those pupils in receipt of free schools meals (FSM) or who have been 
recipients at any given point in the previous six years (EverFSM). A small number of 
looked after and adopted children are also classed as disadvantaged but do not feature in 
this study. 


Reducing the difference in achievement between disadvantaged pupils and their more 
advantaged peers is a focus for policy in all parts of the UK. The Social Mobility 
Commission is an independent body established to assess progress in improving social 
mobility in England, Wales and Scotland. In their 2019 State of the Nation report they 
found that social mobility had stagnated in England over the previous four years, but 
improved in Wales (Social Mobility Commission, 2019). The Equality Commission for 
Northern Ireland issued a statement on educational inequality in October 2017 which 
concluded that ’despite the fact that overall levels of attainment are increasing, many of 
the attainment gaps between the equality groups are also increasing’ (Equality 
Commission, 2017, page 3). 


In England, there is evidence that the disadvantage gap stopped closing between 2015 
and 2019 and there are indications this gap has widened (Hutchinson et al., 2020). In July 
2020 the Education Minister in Northern Ireland announced the appointment of an expert 
panel to examine the links between educational underachievement and social 
disadvantage. An online survey was launched in September 2020 engaging various 
stakeholders, with the aim ‘To examine links between persistent educational 
underachievement and socio-economic background’ (Department of Education, 2020). 
The Welsh Government has announced plans to commence its socio-economic duty in 
March 2021, which will place a legal requirement on specific public bodies to consider how 
their decisions might help to reduce the inequalities associated with socio-economic 
disadvantage (Welsh Government, 2020). 


The COVID-19 pandemic is likely to have a disproportionate impact on economically 
disadvantaged pupils (Children’s Commissioner 2020; Montacute, 2020; Sharp et al., 
2020) and the Education Endowment Foundation estimates that school closures in 
England are likely to have widened the attainment gap between disadvantaged pupils and 
their peers, reversing the progress since 2011 (EEF, 2020a). Whilst the data in this report 
was collected prior to the COVID-19 pandemic, the findings, in combination with other 
research, could be used to strengthen the support given to disadvantaged pupils in an 
attempt to lessen the impact of the pandemic on their future. 


2.1 Which policies have supported the PISA 2018 generation 
by aiming to improve attainment outcomes for 
disadvantaged groups? 


The PISA 2018 cohort in England, Northern Ireland and Wales, born in 2002/03, has 
experienced many policies with the overarching aim of improving attainment for all pupils, 


as shown in Table 1. 


Table 1 Education policy timeline for the PISA 2018 generation 


School | Cohort England events Northern Ireland Wales events 
year age in events 
autumn 
2003 0 National Literacy School self-evaluation: | 
Strategy (from 1997) |Together Towards 
London Challenge Improvement | 
launched May 2003 
2004 1 - - Key stage tests 
discontinued 
2005 2 Free early education forEstablished the - 
all 3-year-olds Common Funding 
Scheme 
2006 3 Rose Report Sure Start widened to }+ 
recommends cover education 
systematic synthetic outcomes 
phonics Extended Schools 
launched 
2007 4 City Challenge Northern Ireland - 
launched April 2008 curriculum introduced 
2008 5 National roll-out Every |Every School aGood  - 
Child a Reader School April 2009 
KS3 tests discontinued 
2009 - - - 
2010 [Academy acceleration | - 
Commitment to expand 
Teach First & school- 
led Initial Teacher 
Training 


School | Cohort England events Northern Ireland Wales events 
year age in events 
autumn 
2011 8 Pupil Premium Count, Read: Succeed |Welsh-Medium 
introduced Review of GCSE policy Education Strategy April 
Education Endowment 2010 
Foundation established School banding 
EBacc performance introduced 
measure introduced 
2012 9 Pupil Premium [Assessment of Regional consortia 
extended to ever-6 Communication using |working from Sept 2012 
Year 7 catch-up Levels of Progression |Standardised tests in 
premium introduced __ |introduced reading and numeracy 
Understanding from May 2013 
Difficulties in Literacy [Pupil Deprivation Grant 
Development: introduced 
Continuing Professional 
Development 2012-15 
2013 10 - Literacy & Numeracy National Literacy and 
Signature Programme (Numeracy Framework 
2013-15 statutory September 
Promoting Improvement2013 
in English and 
Mathematics 2013-15 
2014 11 Revised national - - 
Curriculum for most 
subjects 
Maths Hubs 
established 
2015 12 Reformed GCSEs first | Revised programmes of 
teaching study for English, Welsh 
DfE Single and maths 
Departmental plan 
2015 (withdrawn 2017) 
2016 13 Revised national Policy on Qualifications |- 


curriculum for English 


market and GCSE 
Grading 
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School | Cohort England events Northern Ireland Wales events 
year age in events 
autumn 


came into force for all 
year groups 
Opportunity areas 


launched 

2017 14 Progress 8 - Welsh in Education 
performance measure Action Plan December 
Social mobility plan 2017 


Strategic School 
Improvement Fund (to 
2018) 

Teaching and 
Leadership Innovation 
Fund (to 2020) 
Rollout of secondary 
teaching for mastery of 
maths begins (total 
planned reach 1,700 
schools by 2023) 


Education is an area of devolved government in the UK, with each country making 
decisions on the education system independently. We provide a spotlight on each country 
below, identifying the key policy initiatives which focus more specifically on improving 
attainment outcomes for disadvantaged groups during the PISA 2018 cohort’s experience 
of primary and secondary schooling. The policies largely target literacy and numeracy, as 
these underpin attainment in other areas of learning. 


While many policies aim to tackle socio-economic disadvantage, it is important to note that 
there are other characteristics which are disproportionately associated with disadvantage 
and also lower attainment, for example, special educational needs and some ethnic 
backgrounds. Findings from PISA 2018 (OECD, 2019e) illustrate other factors which could 
affect attainment. For example, UK school leaders of disadvantaged schools were more 
likely than their counterparts in more affluent schools to report that the capacity to provide 
teaching was hindered, at least to some extent, by a lack of teaching staff. 


With regards to early childhood (from age 2 upwards), this cohort in all three countries 
would have had access to the support offered by Sure Start, an initiative aimed at families 
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living in disadvantaged areas by supporting various aspects of preschool children’s 
development including learning, social and emotional skills and health and well-being. 


England: Pupil Premium, EEF and building on the Opportunity Areas 


The Pupil Premium, introduced in 2011 when this cohort was aged 8/9, is funding provided 
to publically funded schools in England to support the education of disadvantaged pupils. It 
comprises a payment per pupil eligible for FSM in year groups from Reception to Year 11. 
The Department for Education (DfE) stated it expected to achieve a significant positive 
impact on the attainment gap in primary schools by 2015, and in secondary schools by 
2020 (National Audit Office, 2015), although more recently it has been called into question 
whether this funding is being used as intended (for example, Education Select Committee, 
2019). 


The Education Endowment Foundation (EEF) was established in 2011 alongside the 
introduction of the Pupil Premium. It aims to identify, fund and evaluate educational 
innovations which meet disadvantaged pupils’ needs. This involves establishing evidence 
of effective interventions and strategies, which work at scale, and encouraging schools, 
government, charities and others to apply this evidence. The EEF has funded projects 
that, after rigorous evaluation, have shown the potential to improve pupils’ attainment are 
termed Promising Projects (EEF, 2020b). These projects (including, for example, peer 
tutoring), have enabled pupils to make +3 months of additional progress in a year, with 
some projects particularly benefitting pupils eligible for FSM, who made on average +4 
months’ additional progress. These programmes are mainly being delivered in primary 
schools, but include one-to-one tutoring and embedding formative assessment at Key 
Stages 3 and 4. 


In 2017, DfE published a social mobility plan, Unlocking Talent, Fulfilling Potential (DfE, 
2017). In the more recent State of the Nation report, the Social Mobility Commission said 
‘In December 2017 the DfE took a concrete step towards recognising the critical 
importance of the role of education in social mobility by publishing its plan for social 
mobility through education. This was the first time there was a meaningful and tangible 
commitment to social mobility in the form of a department-wide plan’ (Social Mobility 
Commission, 2019, page 36). This plan builds on the Opportunities Areas programme, in 
which extra funding was made available to 12 local authority district areas (LADs), 
targeted because of the social, economic and cultural challenges they faced in improving 
people’s life chances. The aim of this programme is to build young people’s knowledge 
and skills and provide them with the best advice and opportunities. The PISA cohort would 
have had a maximum of one year exposure to schemes supported under this programme. 
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Northern Ireland: Common Funding Scheme, the Delivering Social Change Literacy 
and Numeracy Programme and Count, Read: Succeed 


The Common Funding Scheme (CFS), established in 2005 when the PISA 2018 cohort 
were aged 2, is distributed to schools based on the proportion of children entitled to free 
school meals. The Targeting Social Need (TSN) component in the CFS, introduced in 
2014, provides an extra payment to schools to recognise the additional costs in educating 
children from socially deprived backgrounds, regardless of ability, and the particular 
challenges faced in schools with high proportions of disadvantaged children. 


In 2012, Northern Ireland commenced the Delivering Social Change Literacy and 
Numeracy Programme, which aimed to improve literacy and numeracy as part of a wider 
government initiative tackling poverty and social exclusion. The PISA 2018 cohort was in 
their final year of primary school in 2013-14 and so would have experienced the full extent 
of this programme. It aimed to increase the number of pupils in primary and post-primary 
schools achieving at the expected level or above at the end of Key Stage 2 in both Literacy 
and Numeracy and at least a grade C or above in GCSE English and GCSE Mathematics. 


Other policies which may have had an impact include Count, Read: Succeed: which was 
introduced in 2011 and focuses on narrowing gaps in educational outcomes, including 
between the most and least disadvantaged pupils; and Extended Schools, which, from 
2006, has targeted disadvantaged communities. Extended Schools aims to enable the 
provision of more than 3,000 additional services and activities across 500 schools, 
reaching approximately 43% of all schools and 63% of all pupils entitled to FSM in 
Northern Ireland. 


Wales: Pupil Deprivation Grant and National Literacy and Numeracy Framework 


The Pupil Deprivation Grant, launched in 2012, when the PISA 2018 cohort was in its final 
year of primary school, is one of the Welsh Government's flagship education policies and 
aims to tackle the impact of deprivation and disadvantage on educational outcomes. Extra 
funds are available to schools based on the number of pupils eligible for FSM on their roll 
from Years 1-11. The National Literacy and Numeracy Framework, introduced in 2013 
aims to reduce the gaps between pupils with special educational needs or from 
disadvantaged backgrounds, and their peers. 
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2.2 What can we learn from PISA? 


International surveys provide a unique opportunity to examine the 
impact of disadvantage on pupils in England, Northern Ireland and 
Wales 


The release of PISA 2018 data in December 2019 provides rich data to look at the 
reading, maths and science performance of pupils across countries, over time. 
International large scale assessments, such as PISA, give the opportunity for international 
comparisons, as well as UK-wide comparisons on the devolved matter of education. PISA 
measures attainment alongside contextual information, for instance background 
characteristics, from pupil and school questionnaires. 


For PISA, the OECD has developed an alternative measure of socio-economic 
background, the index of economic, social and cultural status (ESCS). ESCS is calculated 
as a weighted average of three indices, generated from the PISA questionnaires: 


e HISEI — the highest occupational status of a pupil’s parents. 


e PAREDINT -— the highest educational level of a pupil’s parents (converted into years 
of education). 


e HOMEPOS - an index of home possessions (measures the availability of 25 
household items that act as measurements of family wealth including the number of 
books at home). 


The index is set to a mean of zero across OECD countries, with a standard deviation of 
one. More detailed information about how the ESCS index is calculated is provided in the 
PISA technical report. 
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How is socio-economic status defined in this report? 


For the purposes of this report we use the Economic, Social and Cultural Status 
(ESCS) index as a measure of pupils’ socio-economic status. A score on the ESCS 
index is estimated for every pupil who participates in PISA. ESCS is based on pupils’ 
responses to questions about their parents’ background and education, and 
possessions in their homes. ESCS is a relative measure. 


How is disadvantage measured in this report? 


Pupils who are amongst the bottom 33% on the ESCS index in their country are 
considered disadvantaged. 


This report, therefore, uses a relative measure of disadvantage within each country, 
and those considered disadvantaged in a more economically advanced country may 
nevertheless have a higher ECSC index than many pupils in less economically 
advanced countries. 


The PISA ESCS index provides a more sophisticated measure of disadvantage than the 
binary ‘FSM or not’ by measuring socio-economic status on a continuous scale. ESCS 
takes sociocultural factors into account (Andrade, 2016) and is also available for pupils 
from every participating PISA country, offering a means for international comparison. 


The overlap between pupils who are FSM eligible and amongst the bottom 33% on the 
ESCS index is presented in Table 2. 


Table 2 Proportion of FSM eligible pupils who are amongst the bottom 33% on the 
ESCS Index 


Country England Northern Ireland Wales 


% 65 61 67 


Socio-economic background has a smaller impact in Macao (China), 
Estonia and Canada than in other high-achieving countries such as 
Singapore 


‘There is no country in the world that can yet claim to have entirely 
eliminated socioeconomic inequalities in education’ - OECD (2018, 
page 13) 
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To gain an accurate picture of interactions between attainment scores and the ESCS 
index, we can look at the amount of variance in scores which can be explained by socio- 
economic background, or the strength of the effect. This shows the extent to which the 
scores of pupils in each country are predicted by socio-economic background, rather than 
by other variables. Figure 1 shows the strength of the effect that ESCS has on reading 
performance for all participating countries. 


Figure 1 Strength of the relationship between performance and ESCS for all PISA 
2018 countries 


Countries such as Macao (China), Estonia and Canada have weaker 
links between socio-economic background and outcomes 
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Source: PISA 2018 database 


In PISA 2018, countries varied in the impact of socio-economic status on success of 
education across countries (OECD, 2019d). As indicated in the graph above, ESCS has 
less of an impact on reading performance in countries such as Macao (China), Estonia 
and Canada than it does in other high-achieving countries such as Singapore. The 
strength of the relationship varies across England, Northern Ireland and Wales and this will 
be discussed further in Chapter 3. 


In the following analyses, some comparisons are made to Estonia and Canada as high- 
achieving countries with greater equity. In PISA 2018, both countries had smaller socio- 
economic gaps in reading (that is, the difference between the average reading scores for 
advantaged and disadvantaged pupils) compared with most other countries (OECD, 
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2019c). Estonia and Canada also significantly? out-performed England, Northern Ireland 
and Wales in all three subjects (Sizmur et a/., 2019a; Sizmur et a/., 2019b; Sizmur et al., 
2019c). Comparisons will also be drawn between England, Northern Ireland and Wales, 
and also the Republic of Ireland where relevant to complement the simultaneous analysis 
PISA 2018 additional analyses: Learning from other countries (Sizmur et al., forthcoming). 


2 When statistical significance is reported, it indicates that the compared averages are significantly different 
at the 5% level. 
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3 How well are England, Northern Ireland and Wales 
supporting the performance of disadvantaged 
pupils? 


3.1 How are disadvantaged pupils in England, Northern 
Ireland and Wales performing compared with their peers 
internationally? 


The gap in performance between the most and least disadvantaged 
pupils was significantly greater in England than in Wales for reading, 
maths and science 


Figure 2 shows? that, as expected, more advantaged pupils performed better in reading 
than their disadvantaged peers in England, Wales and Northern Ireland. Despite similar 
reading scores for the most disadvantaged pupils in all three countries, the gap between 
the most and least disadvantaged pupils was largest in England. There was a 100 score 
point difference in achievement of the most and least disadvantaged pupils in England, 
significantly larger than in Wales (71 score points). The gap of 88 score points in Northern 
Ireland was not significantly different from England and Wales. Advantaged pupils tend to 
do better in England and Northern Ireland than in Wales, with a notable difference of 
scores from Wales emerging by the 3% decile in England and the 6" decile in Northern 
Ireland. Overall, therefore, there is greater equity in Wales than in England. 


3 By splitting pupils into ten equal groups (deciles) according to their ESCS score. 
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Figure 2 Average reading score at each ESCS decile in England, Northern Ireland 
and Wales 


In England, Northern Ireland and Wales, more 
advantaged pupils performed better in reading 
than their disadvantaged peers 
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There was also greater equity in Wales compared to England in maths and science 
achievement, as shown in Figures 3 and 4. The differences in scores between the most 
and least disadvantaged pupils in England (103 score points for maths, 105 score points 
for science) were significantly higher than in Wales (76 score points for maths, 77 score 
points for science), and not significantly different from Northern Ireland (89 score points for 
maths, 87 score points for science). 
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Figure 3 Average maths score at each ESCS decile in England, Northern Ireland and 
Wales 


England had a greater difference in average maths 
scores between advantaged and disadvantaged 
pupils 
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Figure 4 Average science score at each ESCS decile in England, Northern Ireland 
and Wales 


England had a greater difference in average 
science scores between advantaged and 
disadvantaged pupils 
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Socio-economic background had a smaller impact amongst some of the 
most disadvantaged pupils than for their more advantaged peers 


The difference in performance between the most- and least- disadvantaged pupils, as 
noted above, was driven by a higher performance among the most advantaged pupils. 
This is illustrated in Figures 2 to 4 above, where the line is flatter at the lower levels of 
ESCS compared with higher levels. For example, in reading, there were smaller increases 
in score between the 1St and 5" deciles in all three countries (44 in England, 25 in Northern 
lreland and 27 in Wales) compared with the greater increase between the 5" and 10! 
deciles (56 in England, 63 in Northern Ireland and 44 in Wales). 


In Northern Ireland, there was very little difference in the score for pupils between the 2" 
to 5" decile, which was not seen in Wales or England. This pattern was mirrored for 
Estonia, a high-achieving comparator country with high equity, between the 2™ and 4" 
deciles in all three subjects, as illustrated for science in Figure 5. 


Figure 5 Average science score at each ESCS decile in Estonia and Northern Ireland 


There was little difference in scores amongst 
pupils at the lower end of the ESCS index in both 
Estonia and Northern Ireland 
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The smaller difference in scores for the most socio-economically disadvantaged pupils 
suggests a threshold (after the 5‘ decile) for which increases in socio-economic 
background are reflected in higher gains in performance. Prior to this threshold (below the 
56 decile) there is little difference in average attainment between one pupil and a 
marginally more affluent peer. This indicates that socio-economic status is having different 
effects on these two groups of pupils in terms of attainment. It may be that below the 
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threshold (below the 5" decile), there is less variation (a more equal picture), as the most 
disadvantage pupils are performing better than expected. Or, it may be, that at each stage 
along the ESCS index above this threshold, non-disadvantaged pupils are performing 
even better than expected leading to more variation in attainment, indicated by steeper 
line. This effect warrants further investigation. 


In England, more advantaged pupils perform similarly to their peers in 
Canada and Estonia 


In England, pupils at the top end of the ESCS index (above the 8" decile) perform similarly 
to those in Estonia and Canada across all three subjects (illustrated below for reading). 
Pupils in England below this decile score, on average, lower than those in Canada and 
Estonia. Therefore, improving the performance of disadvantaged pupils would reduce the 
significant differences seen in reading between countries (Sizmur et al/., 2019b). 


Figure 6 Average reading score at each ESCS decile in England, Canada and 
Estonia 


In England, more advantaged pupils performed 
similarly to their peers in Canada and Estonia. This 
was not the case for disadvantaged pupils 
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Socio-economic background explains less of the variance in 
achievement in Wales and Northern Ireland compared with the OECD 
average 


As noted in Chapter 2, to gain an accurate picture of interactions between scores and the 
ESCS Index, we can look at the amount of variance in scores which can be explained 
by socio-economic background, or the strength of the effect. This shows the extent to 
which the scores of pupils in each country are predicted by socio-economic background, 
rather than by other variables. 


In common with other PISA countries, socio-economic status is associated with attainment 
in all three subjects and across England, Northern Ireland and Wales. For example, across 
the OECD, on average, 12% of the variance in reading scores can be explained by socio- 
economic background. The amount of variance in scores for the OECD average was more 
than Northern Ireland (7%) and Wales (4%) but not statistically different from England 
(10%) (Sizmur et a/., 2019a; Sizmur et al., 2019b; Sizmur et a/., 2019c). 


This indicates that there are other variables which contribute to achievement and that 
socio-economic background can only explain some of the differences. In Wales and 
Northern Ireland, socio-economic background explains less, on average, than in other 
countries thereby suggesting that there are other factors at play which are contributing 
more to the differences in achievement. 


3.2 How does this compare with previous years? 


The following section presents the average reading, maths and science scores for the 
disadvantaged pupils (those pupils amongst the bottom 33% on the ESCS index) and the 
rest of the sample (those amongst the top 67% on the ESCS index) across PISA cycles. 


In England, Northern Ireland and Wales, the 2018 average scores for 
disadvantaged pupils in maths and reading were significantly higher 
than in previous cycles 


Pupils who participated in PISA 2018 had experienced some of the country-led initiatives 
to improve both literacy and numeracy, such as the EEF programmes in England; Count 
Read: Succeed in Northern Ireland and the National Literacy and Numeracy Framework in 
Wales. It is, therefore, encouraging that in England, Northern Ireland and Wales there was 
evidence of improvement amongst disadvantaged pupils since previous PISA cycles for 
reading and mathematics. 


In England, Northern Ireland and Wales, average scores in reading and mathematics for 
disadvantaged pupils (bottom 33%) were significantly higher in 2018 than in at least one 
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previous cycle. Significant changes in England for disadvantaged pupils were seen 
between 2009 and 2018 for reading and maths. There was also a significant improvement 
in maths between 2015 and 2018 for non-disadvantaged pupils (top 67%), whilst the 
improvement for disadvantaged pupils was not significant. 


In Northern Ireland disadvantaged pupils scored significantly higher in reading in 2018 
than in 2009 and in 2012 and significantly higher in maths than in 2012. There was no 
significant change for non-disadvantaged pupils in Northern Ireland in 2018 compared to 
previous cycles. Wales followed a similar pattern in both reading and maths in that both 
the 2009 and 2012 scores for disadvantaged pupils had significantly improved in 2018. 
General performance in Wales for mathematics had improved for both groups since 2009 
and 2012, as presented in Figure 8, which shows an upward trend for both the 
disadvantaged and non-disadvantaged pupils. 
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Figure 7 Average PISA reading scores for disadvantaged pupils and non- 
disadvantaged pupils over time in England, Northern Ireland and Wales 


In Northern Ireland, the 2018 average reading score for the 


In England, the 2018 average reading score for the bottom 2 
bottom 33% ESCS was significantly higher than in 2009 and 
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Figure 8 Average PISA maths scores for disadvantaged pupils and non- 
disadvantaged pupils over time in England, Northern Ireland and Wales 


In England, the 2018 average maths score for the bottom 33% 
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A few examples from comparator countries are included in Figure 9, where the picture is 
more mixed. Maths performance in Canada for both disadvantaged and non- 
disadvantaged pupils significantly declined between 2009 and 2018; for reading it has 
remained stable. Whilst there has been improvement for non-disadvantaged pupils in 
Estonia for maths and reading, for disadvantaged pupils it was only the 2009 average 
reading score that was significantly different from 2018, a result of the 2009 to 2012 rise, 
with scores remaining stable between 2012 and 2018. Reading performance for 
disadvantaged and non-disadvantaged pupils in the Republic of Ireland has followed a 
similar pattern for both groups, with 2009 scores that were significantly lower than in 2018. 
However, the Republic of Ireland’s 2009 overall scores in reading and maths have often 
been reported as outliers (for example, McKeown et al., 2019) due in part to the 
combination of some demographic changes, issues with the reading literacy test design, a 
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drop-off in pupils’ engagement with PISA, and the international scaling procedure used in 
2009 (Cosgrove & Cartwright, 2014). 


Figure 9 A spotlight on average reading and maths scores for disadvantaged and 
non-disadvantaged pupils across comparator countries 
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Science performance amongst disadvantaged pupils remained stable 
over time in England, Northern Ireland and Wales 


There have been several policies introduced specifically to raise disadvantage pupils’ 
attainment in literacy and maths (see Chapter 2). Schools may also be using policies such 
as Pupil premium, the Common funding scheme and the Pupil Deprivation Grant to further 
focus in on these areas. Therefore, it is perhaps unsurprising that whilst the most recent 
performance in reading and maths of disadvantaged pupils improved since previous 
cycles, performance in science has remained stable over this period. Figure 10 illustrates 
that average scores for disadvantaged pupils have not significantly changed in England, 
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Northern Ireland and Wales. The only significant changes seen across cycles was for non- 
disadvantaged pupils in Northern Ireland, that is a significant decrease in 2018 in average 
science score from all three previous PISA cycles. Overall science performance 
significantly declined in Northern Ireland over the same period (Sizmur et a/., 2019a) so it 
is somewhat encouraging that this overall pattern is not reflected in the scores for 
disadvantaged pupils. 


Figure 10 Average PISA science scores for disadvantaged pupils and non- 
disadvantaged pupils over time in England, Northern Ireland and Wales 


In England, the average score for science in 2018 was not In Northern Ireland, the average score for science for the top 
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Whilst the 2018 science scores are not significantly different in England, Northern Ireland 
and Wales for disadvantaged pupils compared to previous cycles, the picture across 
comparator countries was different. In Canada and the Republic of Ireland the 2018 
average science scores for both disadvantaged and non-disadvantaged pupils were 
significantly lower than the average scores from at least one previous cycle, as illustrated 
with Canada in Figure 11. For Estonia, the 2018 average score for disadvantaged pupils 
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was significantly lower than in 2012 and there was no change for non-disadvantaged 
pupils. This pattern across comparator countries reflects a decline in performance, on 
average, across OECD countries since 2012, the reason for which still warrants further 
investigation. 


Figure 11 Average PISA science scores for disadvantaged pupils and non- 
disadvantaged pupils over time in Canada 


In Canada, the 2018 average science scores for both 
disadvantaged and non-disadvantaged pupils were lower 
than in at least one previous cycle 
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3.3 Are there areas of reading at which disadvantaged pupils 
are particularly weak? 


The main focus of PISA 2018 was reading, which means that we can look at performance 
in reading in more depth. PISA assess three reading processes which readers use when 
engaging with texts. These were ‘locating information’, ‘understanding’ and ‘evaluating and 
reflecting’. Figure 12 shows the average performance in each of these processes for 
disadvantaged pupils (bottom 33% ESCS, bars) compared with their non-disadvantaged 
peers (top 67% ESCS, rhombus). 


Within each country, there was a similar gap between disadvantaged 
pupils and their non-disadvantaged peers for all three reading 
processes 


Non-disadvantaged pupils scored significantly higher than disadvantaged pupils on all 
three reading processes in England, Northern Ireland and Wales. The gap between these 
two groups of pupils for each reading process was similar within each country. This 
suggests a need for an improvement in disadvantaged pupils’ performance across all three 
reading processes, rather than just one, in order to decrease the significant gap between 
disadvantaged pupils and their non-disadvantaged peers. 


Scores for disadvantaged pupils were the lowest in the process of understanding for all 
three countries. However, that is not to say they are disproportionately weaker in this 
process, as generally, the overall country averages for the process understanding were 
lower than for the other two processes (Sizmur et al/., 2019a; Sizmur et al., 2019b; Sizmur 
et al., 2019c). The largest gap between disadvantaged pupils and their peers was in 
evaluating and reflecting, however this is unsurprising as this is the highest-level process 
assessed for PISA in reading. 
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Figure 12 Disadvantaged pupils’ performance in the three PISA reading processes 
in England, Northern Ireland and Wales 


Disadvantaged pupils did significantly less well in all three 
reading processes compared to their advantaged peers 
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These findings were mirrored across Estonia, Canada and the Republic of Ireland, as 
shown in Figure 13. Disadvantaged pupils in Estonia appear to do better in locating 
information in comparison with the two other reading processes. 
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Figure 13 Disadvantaged pupils’ performance in the three PISA reading processes 
in the Republic of Ireland, Canada and Estonia 


Across comparator countries, disadvantaged pupils also 
scored significantly lower than their peers in all three reading 
processes 
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4 Resilience 


4.1 How is resilience defined? 


Resilient pupils are those disadvantaged pupils who, despite the odds, 
are equipped with the skills they need to succeed in later life 


In spite of socio-economic disadvantage, some pupils overcome barriers and attain high 
levels of academic proficiency. These pupils are known as ‘resilient’ pupils (OECD, 
2019d). Previous research suggests that factors such as perceiving a more positive school 
climate, not seeing intelligence as fixed (that is, having a growth mind-set) and receiving 
support from parents are associated with resilience in some countries (OECD, 2019d). 
Further analysis from NFER has found that resilience was associated with factors such as 
self-confidence (Wheater et a/., 2016), and having a good attendance at school (Wheater 
et al., 2016; Bradshaw et al., 2018). It is worth noting that previous research into resilience 
has used alternative definitions of resilience than the one used for this report (for example, 
OECD, 2019c)’. 


How is ‘resilience’ defined in this report? 


Pupils that are among the 33% most socio-economically disadvantaged pupils in their 
country but are able to achieve at or above “level 3” in all three PISA domains. 


This alternative definition, based on an OECD working paper (Agasisti et a/., 2018), 
provides an absolute and objective criteria of achievement (proficiency at or above Level 
3). Additionally, by achieving a level 3 or above in all three PISA domains, resilient pupils 
are able to demonstrate they have acquired the skills they need to succeed in later life 
(Agasisti et a/., 2018). 


4 For PISA 2018, OECD defined resilience as those pupils who are in the bottom quarter of the PISA index 
of ESCS in their own country / economy but who score in the top quarter of reading in that country / 
economy. Resilience has previously been defined as high-achieving in one subject rather than being high- 
achieving across all three. 
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What is Level 3 in PISA? 


PISA uses proficiency levels to describe the types of skills that pupils are likely to 
demonstrate and the tasks that they are able to complete (OECD, 2019b). For 
example, pupils performing at a level 3 can integrate content and generate both basic 
and more advanced inferences in reading; in maths they show some ability to handle 
percentages, fractions and decimal numbers, and to work with proportional 
relationships; in science they can draw upon moderately complex content knowledge to 
identify or construct explanations of familiar phenomena. 


PISA measures pupils’ ability to apply their knowledge to solve problems in real-world 
situations and is not based on specific curriculum content in participating countries. 
Further analysis to equate PISA scores with GCSE grades in English and mathematics 
is out of scope of this research. See Gambhir et a/. (2020) or Jerrim & Shure (2017) for 
analysis which compares PISA results with GCSE grades. 


Around a third of disadvantaged pupils in England, Northern Ireland and 
Wales were resilient 


In England, Northern Ireland and Wales, around a third of disadvantaged pupils in PISA 
2018 could be defined as resilient (as illustrated in Figure 14). This means around one- 
third of disadvantaged pupils achieved a level 3 or above in all three PISA domains. The 
OECD average for resilient pupils was similar to England, Northern Ireland and Wales 
(34%). If using the traditional OECD definition of resilience, the UK tends to have an above 
average proportion of disadvantaged pupils who are considered resilient. For example, in 
PISA 2018, 14% of disadvantaged pupils in the UK were considered resilient in reading 
(that is, they were amongst the bottom quarter on the ESCS index but scored in the top 
quarter of reading performance in their own country), compared to an OECD average of 
11% (OECD, 2019c). 


The proportions of resilient pupils in the Republic of Ireland (33%) and Canada (36%) were 
also similar to England, Northern Ireland and Wales. There were greater proportions of 
resilient pupils in Estonia (44%). 
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Figure 14 Proportion of resilient pupils across countries 
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The subsequent sections explore the characteristics which distinguish resilient pupils from 
their peers. Two types of analyses have been conducted: profiling and a multilevel logistic 
regression. The results from these analyses are presented below. 


4.2 How do resilient pupils’ attitudes differ from similarly 
performing pupils from more advantaged backgrounds? 


Variables associated with wealth explain the main differences between 
disadvantaged resilient pupils and non-disadvantaged pupils who 
achieved level 3 or higher in all three subjects 


We conducted profiling between resilient pupils (disadvantaged and high-achieving pupils) 
and their similarly performing peers from more advantaged backgrounds (that is pupils 
within the top 67% on the ESCS index who achieve a level 3 or above in all three PISA 
domains). Derived variables from the pupil questionnaire were included in this analysis as 
well as single questions which had been associated with the previous definition of 
resilience in the literature (the rationale behind the inclusion of variables in this analysis 
can be found in Appendix A). 
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What is profiling? 


Profiling is based on bivariate correlation tests; exploring each variable’s association 
with resilience without controlling for other variables. 


Variables which were correlated above +/-0.15 are presented below. 


In all three nations, variables associated with wealth were highly correlated (above +/- 
0.15) with resilience and therefore could be seen as differentiating between non- 
disadvantaged pupils who achieved a level 3 and above in all three PISA domains and 
resilient disadvantaged pupils. This included differences across variables such as parental 
occupation, possessions in the home, home educational resources and ICT resources in 
the home?. These findings were consistent across England, Northern Ireland and Wales. 
Whilst wealth being a distinguishing factor between the two groups is not surprising, as by 
definition, resilient pupils are less affluent, it is interesting that there were no differences in 
attitudes between the two groups, except in Northern Ireland. 


In Northern Ireland, resilient pupils were less likely be confident in 
reading compared to their more affluent, high-achieving peers 


Pupils were asked how much they agreed or disagreed with the following statements: | am 
a good reader; | am able to understand difficult texts; | read fluently. In Northern Ireland, 
resilient pupils were less likely to respond as positively to these statements than their 
similarly performing peers from more affluent backgrounds. This was not seen in England 
and Wales. 


There was little difference between resilient pupils’ attitudes compared 
with their non-disadvantaged, high-achieving peers 


In the profiling analyses, the majority of the attitudinal variables did not show up as being 
different (that is, they did not correlate above +/-0.15) between disadvantaged pupils and 
their advantaged, high-achieving peers. This included how much they enjoyed/liked 

reading, how satisfied they were with their life and their perceived fear of failure. In terms 
of the school environment, there were no differences in their experiences of being bullied, 
their sense of belonging in school, their perception of competitiveness and cooperation at 
school and the perceived support from their teachers in their English lessons. Additionally 


5 Only available for England as variable derived from the ICT questionnaire which was not administered in 
Wales or Northern Ireland. 
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there was no difference between pupils’ aspirations for further education and future 
careers. This reinforces that it was largely socio-economic status that distinguished the 
two groups. 


4.3 How do resilient pupils’ attitudes differ from low- 
performing pupils from similar backgrounds? 


We compared resilient pupils with their disadvantaged peers who do not perform at or 
above a level 3 in all three subjects. 


The profiling analyses revealed that compared to their disadvantaged low-achieving peers, 
resilient pupils in England, Northern Ireland and Wales were more likely to: 


e use metacognitive strategies (such as such as summarising texts, assessing 
credibility; and understanding and evaluating) 


e like reading 
e have self-confidence 
e have high aspirations for their future careers 


e have a growth mind-set (that is, seeing your intelligence as something that is not 
fixed) 


e report they would invest a lot of effort if something was important to them 
And less likely to: 
e report finding the PISA test difficult 


Some findings were not consistent across all three countries. These are presented in 
Table 3, where a ‘yes’ or a ‘no’ indicates where differences were found between resilient 
pupils and their disadvantaged peers. 
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Table 3 Country-specific findings from the profiling analyses 


Finding England Northern Wales 
Ireland 

Resilient pupils were less likely to report Yes No Yes 
finding reading difficult. 
Resilient pupils spent more time reading No Yes Yes 
for enjoyment 
Resilient pupils reported less disruption in | No Yes No 
their English/Welsh® lessons. 
Resilient pupils were less likely to report No Yes No 
their life has meaning and purpose. 
Resilient pupils were less likely to have No No Yes 
skipped a whole day of school in the last 
two weeks. 
Resilient pupils were more likely to report | Yes No No 
having invested effort into the PISA test. 


Source: PISA 2018 dataset 


It is worth noting that the profiling did not find a difference between the two groups with 
regards to pupils’ ESCS for all three countries. This means that out of all disadvantaged 


pupils, the low-achieving pupils, on average, were not any more disadvantaged that their 


resilient peers. 


In this instance, profiling was used as an exploratory technique to test each variable’s 
association with resilience without controlling for other variables, so as to suggest a 


hypothesis about what may be driving resilience. The results from the regression (Section 


4.4) further explore these variables by testing the associations of all variables and 


resilience jointly. 


6 Pupils who took the assessment in Welsh were asked about their Welsh lessons. 
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Logistic regression analysis 


To identify factors associated with resilience we undertook three multi-level logistic 
regression analyses (one for England, Northern Ireland and Wales). This predicted the 
likelihood of a disadvantaged pupil being classified as resilient given their individual 
characteristics. The variables which were included in this analysis were those which 
had been identified within the profiling (see section 5.3), or based on previous research 
around resilience, or country-level areas of interest (the rationale behind the inclusion 
of variables in this analysis can be found in Appendix A). 


It is important to note when interpreting these findings that they demonstrate an 
association, but do not prove causality. Indeed, it is possible that the factors identified 
are a consequence, rather than a cause, of higher achievement. Nevertheless, this 
analysis serves to highlight areas for future attention by policymakers, practitioners and 
researchers. 


How are profiling and multi-level logistic regression analyses 
different? 


Profiling is based on bivariate correlation tests, exploring each variable’s association 
with resilience without controlling for other variables. Regressions test the association 
of all variables and resilience jointly. This means that results will differ from those 
obtained in the correlations as only the unique variance explained by each variable will 
remain in the regression. 


Profiling is more of an exploratory technique that can suggest hypotheses about what 
might be driving resilience and the regression results will be more robust and 
confirmatory. Both sets of results are informative which is why we present both. 


4.4 In what circumstances do disadvantaged pupils tend to 
overcome barriers to perform better? 


The variables associated with resilience’ are provided below for each country. The higher 
the degree of significance, identified by the number of stars, the less likely the observed 
result would occur due to chance. The findings are similar across countries but are not an 


7 In this regression ‘resilience’ is the dependant variable. Resilience is a binary variable where 0 is 
disadvantaged but non-resilient pupils and 1 is resilient pupils. 
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exact match. This reflects what is seen generally in PISA results and specifically OECD 
findings around resilience®, which is that the strength of relationships between factors vary 
across countries. 


In comparison to their disadvantaged, but low-achieving peers, resilient pupils in 
England were more likely to: 


e use metacognitive strategies to summarise *** 

e have a higher KS2 average point score *** 

e use metacognitive strategies to assess credibility ** 

e have high expectations for their future qualifications ** 
e put effort into the PISA test ** 


e attend a school where the admission policy is based on feeder schools 
recommendations ** 


e have a growth mind-set * 
And less likely to: 
e have found the PISA test difficult *** 


e report their life has meaning, that they've found a meaning in life and they have a 
clear sense of what gives meaning to their life *** 


e have a high perception of receiving emotional support from parents ** 
e attend a school where the admission policy is residence in a particular area ** 


e have skipped a whole day of school in the last two weeks ** 
*(p < 0.05), **(p < 0.01), ***(p < 0.001) 


Source: PISA 2018 data matched to National Pupil Database 


8 Resilience as defined by the OECD; those pupils who are in the bottom quarter of the ESCS index in their 
own country but who score in the top quarter of reading in that country. 
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In comparison to their disadvantaged, but low-achieving peers, resilient pupils in 
Northern Ireland were more likely to: 


e expect a higher level occupation in the future ** 
e use metacognitive strategies to assess credibility ** 
e see the importance in trying hard at school ** 
e be protestant * 
e report they would invest a lot of effort if something was important to them * 
e have a growth mind-set * 
And less likely to: 
e have found the PISA test difficult ** 
e regularly have positive emotions ** 


*(p < 0.05), **(p < 0.01), ***(p < 0.001) 
Source: PISA 2018 data matched to Northern Ireland School Census data 
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In comparison to their disadvantaged, but low-achieving peers, resilient pupils in Wales 
were more likely to: 


e use metacognitive strategies to assess credibility *** 
e use metacognitive strategies to summarise *** 
e have high expectations for their future qualifications ** 
e expect a higher level occupation in the future ** 
e have a growth mind-set * 
e have self-confidence * 
And less likely to: 
e have found the PISA test difficult *** 
e have taken the test in Welsh *** 
e have skipped a whole day of school in the last two weeks ** 
e regularly have positive emotions ** 
e attend a Welsh-medium school ** 


e report their life has meaning, that they’ve found a meaning in life and they have a 
clear sense of what gives meaning to their life ** 


e have disruptions in their English/Welsh® lessons * 
*(p < 0.05), **(p < 0.01), ***(p < 0.001) 
Source: PISA 2018 data matched to National Pupil Database 


The findings above present a profile of a resilient pupil in England, Northern Ireland and 
Wales. The section below explores findings which are true for at least two of these 
countries and concludes by discussing the associations between resilience and country- 
specific variables. 


° Pupils who took the assessment in Welsh were asked about their Welsh lessons. 
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Resilient pupils in England, Northern Ireland and Wales were more likely 
to use metacognitive strategies to assess the credibility of information 


Resilient pupils in England and Wales were more likely to use 
metacognitive strategies to summarise text 


Metacognition is an awareness of how one develops an understanding of text and uses 
reading strategies (OECD, 2019a), or more simply, it is the means in which pupils can 
learn about how they learn. Questions were included in the pupil questionnaire on three 
aspects of metacognition: summarising; assessing credibility; and understanding and 
evaluating. In England, Northern Ireland and Wales, there was an association between 
resilience and using metacognition strategies to assess credibility. In England and Wales 
only, using strategies to summarise was also associated with resilience. 


These findings support work by the Education Endowment Foundation to encourage the 
fostering of these strategies through teaching and supportive classroom practices. The 
Sutton-Trust-EEF Teaching and Learning Toolkit rates metacognition as a high-impact, 
low cost approach to improving the attainment of disadvantaged learners and the evidence 
suggests that use of these strategies can be worth the equivalent of an additional seven 
months’ progress (EEF, 2018). 


Resilient pupils were less likely to see intelligence as something that is 
fixed 


Pupils were asked whether they agreed or disagreed with the statement ‘Your intelligence 
is something about you that you cannot change very much’. Pupils with a fixed mind-set 
are less likely to take action to improve their skills if they are deemed unsatisfactory, even 
when there is an awareness that these skills will have implications for future successes 
(Hong et al., 1999). 


In England, Northern Ireland and Wales, resilient pupils were more likely to have a growth 
mind-set than their disadvantaged, but low-achieving peers, that is, they do not see their 
intelligence as something that is fixed. This means resilient pupils recognise challenges as 
external, understanding they can be confronted and tackled. An association between 
resilience and having a growth mind-set was also seen internationally (OECD, 2019d). 


Resilient pupils had high aspirations for their future education and 
future careers 


Resilient pupils had high aspirations compared with their disadvantaged, low-achieving 
peers. This came in the form of high expectations for their future qualifications (England 
and Wales) and high expectations for their future occupation (Wales and Northern Ireland). 
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High aspirations may act as a self-fulfilling prophecy and high aspirations may lead to high 
attainment, but conversely high attainment is likely to encourage pupils to have higher 
aspirations. Regardless of the direction of causality, by 15 years old, it would be expected 
that pupils have a good idea of the general level of jobs and education which are in scope 
for them, so any policies or interventions which focus on raising aspirations should start 
from an earlier age. 


Resilient pupils in England and Wales were, counterintuitively, less 
likely to report having found meaning in life 


Pupils were asked whether they agreed or disagreed with three statements: My life has 
clear meaning or purpose; | have discovered a satisfactory meaning in life; | have a clear 
sense of what gives meaning to my life (forming the eudemonia scale). Resilient pupils in 
England and Wales were less likely to agree with these statements, indicating they were 
less likely to have found meaning in their lives than their disadvantaged, low-achieving 
peers. PISA 2018 showed that internationally, pupils who generally scored higher in 
reading scored lower on the eudemonia (a sense of meaning and purpose in life) scale 
(OECD, 2019d), suggesting that eudemonia is negatively linked to achievement. For 
example, pupils who report having discovered a satisfactory meaning in life are less likely 
to be high-achieving. Kuhn et a/ (2021) found that aspects of wellbeing (life satisfaction, 
positive emotions and eudemonia) were negatively related to achievement. This finding 
could also be linked to other associations such as aspirations, in that pupils who haven’t 
yet achieved all they want to achieve may not feel they have yet found a meaning in their 
life. More research is needed to investigate further the overall, counterintuitively, negative 
relationship between eudemonia and achievement so that policy makers can consider the 
implication of this for disadvantaged pupils. 


Resilient pupils were less likely to have positive emotions in Wales and 
Northern Ireland 


Pupils were asked how often (from never to always) they normally felt a range of emotions 
such as happy, proud, joyful, scared and worried. This made up the subjective wellbeing 
scale. Resilient pupils in Wales and Northern Ireland were less likely to respond positively 
to these question compared with disadvantaged pupils who were not resilient. That is 
resilient pupils may experience positive emotions less frequently and / or experience 
negative emotions more frequently. There is a well-established relationship between 
higher achievement and lower wellbeing in England, Northern Ireland and Wales, as 
reported in the simultaneous analysis on wellbeing (Kuhn et al., 2021). This negative 
relationship might be driving these findings around resilience for Wales and Northern 
Ireland (and the eudemonia findings noted above) as resilient pupils, by definition, 
performed better than their disadvantaged, low-achieving peers. At both the pupil and 
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country level this appears to be a complex relationship and there may currently be a trade- 
off between wellbeing and achievement, further reflected in the differences between 
disadvantaged pupils, that is resilient and non-resilient pupils. 


There was evidence of higher motivation in resilient pupils in Northern 
Ireland and England 


Pupils responded to three statements ‘Trying hard at school will help me get a good job’; 
‘Trying hard at school will help me get into a good college’; and ‘Trying hard at school is 
important’. In Northern Ireland, resilient pupils were more likely to see the importance in 
trying hard at school than their disadvantaged, low-achieving peers. Additionally, resilient 
pupils in England were more likely to report putting more effort into the PISA test. 


Lower levels of truancy was associated with resilience in England and 
Wales 


In England and Wales, resilient pupils were less likely to have skipped a whole day of 
school in the last two weeks, supporting previous findings (Wheater et a/., 2016). A lower 
level of truancy was not associated with resilience in Northern Ireland, and it is worth 
noting there was little difference in levels of truancy as reported by pupils compared with 
England and Wales. 


There may be a variety of reasons why pupils skip school (OECD, 2019d) including caring 
responsibilities, lack of engagement, sickness or a fear of being bullied and so identifying 
these reasons is an important first step in supporting socio-economically disadvantaged 


pupils. 


Exploratory machine learning analysis of resilient pupils supports 
our multi-level logistic analysis 


We conducted exploratory analysis using machine learning to answer questions about 
the characteristics of resilient and non-resilient pupils. This analysis has the advantage 
that many variables can be included, helping to identify characteristics and patterns 
which may not have been considered in the multi-level logistic regression model. The 
analysis supports the associations between resilience and metacognitive strategies, 
pupils’ future study and career aspirations and the perception of the level of difficulty of 
the PISA assessment. More information can be found in an NFER working paper 
(Andrade and Liht, forthcoming). 
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There was no evidence that age or gender were associated with 
resilience 


Gender was not found to be a significant predictor of resilience, despite there being some 
gender differences in performance (particularly in reading) across England, Northern 
Ireland and Wales in PISA 2018. 


Additionally, Wheater et al. (2016) had previously found that age was associated with 
resilience when looking at performance in PISA maths, that is, disadvantaged children 
born in the autumn were more likely to be resilient than their peers who were born in the 
following summer. However, there was no evidence that age was associated with 
resilience in this analysis across all three countries. 


Other noticeably absent associations include life-satisfaction (however, there was an 
association with positive emotions as detailed above) and how much pupils enjoyed/liked 
reading. Additionally, school type was included in the regression for Northern Ireland. 
Despite the selective education system in Northern Ireland, there was no association 
found between resilient pupils and the type of schools they attend. 


Country-specific associations 


Some associations were found with country-specific variables. 


Prior attainment in England 


This analysis included the National Pupil Database variable indicating pupils’ Key Stage 2 
average point score. It would be expected that, on the whole, pupils who were high 
performing at KS2 were also high performing by the age of 15. Therefore, it is unsurprising 
that having a KS2 higher average point score was associated with resilience in England as 
by definition, resilient pupils were high-achieving in PISA. 


Welsh language 


The findings from the multi-level regression model suggest resilient pupils were less likely 
to have taken the PISA assessment in Welsh and less likely to attend a Welsh-medium 
school. Of the PISA 2018 sample, a quarter of pupils in Wales attended Welsh-medium 
schools and of these 798 pupils, just over a half took the test in Welsh (458). Some 
caution, therefore, is advised in interpreting this finding due to the small numbers of 
disadvantaged pupils attending a welsh-medium school / taking the assessment in Welsh 
when you break the sample down further. 


Previous research suggests that the lower performance of Welsh-medium schools is likely 
to be driven by the language of assessment (Classick et a/., 2020), but the relationship 
between Welsh language and performance on PISA is complex and requires further 


46 


investigation. For instance, the length of the reading texts are often longer in Welsh than in 
English and in some questions the word count can be up to 25% higher. Therefore, a 
comparison of the time taken to read the assessments in each language could be an area 
for further investigation. 


Religion — Northern Ireland 


In Northern Ireland, protestant boys who are entitled to free school meals persistently 
underachieve which leads to a lack of progression to further and higher education (Burns 
et al., 2015). For the purpose of the logistic regression, religion was collapsed into two 
values, protestant and non-protestants. In this analysis, protestant disadvantaged pupils 
were more likely to be resilient than non-protestant pupils. This goes against what might 
be expected based on the previous research, however, it could be explained by the 
performance of disadvantaged protestant girls. Whilst the interaction of religion by gender, 
included in the model, was not significant, nearly half of protestant disadvantaged girls in 
the sample were considered resilient (45%). This was a much larger proportion than 
protestant boys (26%), and non-protestant boys (29%) and girls (31%). 


Emotional support from parents — England 


It is worth further exploring the link between resilient pupils in England and parental 
support. Pupils were asked how much they agreed with three statements about this 
academic year: ‘My parents support my educational efforts and achievements’; ‘My 
parents support me when | am facing difficulties at school’; ‘My parents encourage me to 
be confident’. The link between parental support and resilience was mixed across 
countries in PISA 2018 (OECD, 2019c). In some countries, significantly more resilient’? 
pupils were observed amongst those pupils in the top quarter of the index of parents’ 
emotional support but in countries such as Singapore, the Republic of Ireland and across 
the UK there was no significant difference. Our analysis suggested that resilient pupils in 
England were less likely to report receiving emotional support from their parents, and, 
therefore, conversely disadvantaged but non-resilient pupils were more likely to report 
emotional support from their parents. The direction of causality cannot be inferred from this 
analysis so we are unable to determine if lack of parental support drives resilience. It may 
be that parents of resilient pupils do not feel they need to provide as much emotional 
support because their child is achieving well at school in comparison to a parent of a low- 
achieving pupil. 


10 Resilience as defined by the OECD; those pupils who are in the bottom quarter of the ESCS index in their 
own country but who score in the top quarter of reading in that country. 
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5 Conclusions and recommendations 


5.1 Conclusions 


Social mobility and improving performance of disadvantaged groups is a priority for all 
parts of the UK. The impact of socio-economic background on performance in reading, 
maths and science is evident across England, Wales and Northern Ireland but to a varying 
degree. In all three countries, more affluent pupils perform, on average, better than their 
disadvantaged peers. Wales appears to have greater equity amongst pupils than in 
England, that is, there is a significantly smaller difference in the average reading, maths 
and science scores for the most- and least-disadvantaged pupils. Northern Ireland sits 
somewhere in the middle, as equity appears to be similar to both Wales and England. 
However, socio-economic background only explains some of the variance between 
attainment in all three countries, explaining more in England and the OCED, on average, 
than in Wales and Northern Ireland. 


Disadvantaged pupils’ performance in reading and maths significantly improved in 2018 
from at least one previous cycle of PISA and this was found in England, Northern Ireland 
and Wales. This is encouraging, considering the policies in place to tackle disadvantaged 
pupils’ underperformance in maths and reading in all three countries. Disadvantaged 
pupils’ performance in science in the three countries has remained stable over time, whilst 
it has decreased in comparator countries. 


Non-disadvantaged pupils significantly out-performed disadvantaged pupils in all three 
reading processes. In England, Northern Ireland and Wales the gaps between 
disadvantaged pupils and their peers was similar for each process, indicating there is not 
just one process in which disadvantaged pupils are disproportionately weak. Instead, 
disadvantaged pupils need support in raising proficiency across all three processes: 
locating information; understanding; evaluating and reflecting. 


Disadvantaged pupils who perform better than the average, given their socio-economic 
background, tend to use metacognitive strategies, believe intelligence is not fixed and 
have high aspirations for their future education or careers. They are less likely to be truant 
or think that their life has meaning or purpose. In this report, these pupils are known as 
resilient pupils, and they make up around a third of the disadvantaged pupils in England, 
Northern Ireland and Wales. We conclude that disadvantaged pupils may benefit from 
programmes which encourage the use of metacognitive strategies and counter the belief 
that intelligence is fixed. Further exploration is needed to determine how and where 
targeted support on aspirations would be most beneficial for disadvantaged pupils. 
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For the most part, there were no attitudinal differences found between resilient pupils and 
their high-achieving, more affluent peers. For example, no differences were found between 
pupils’ sense of belonging at school, life satisfaction or future aspirations. The exception to 
this was in Northern Ireland, where resilient pupils were less confident in their reading than 
their more affluent peers. Instead, it was mainly indicators of family poverty which 
distinguished these pupils. Whilst it is perhaps unsurprising that wealth indicators 
demonstrate the differences between resilient pupils and those who are not disadvantaged 
and high-achieving, it does emphasise that households need sufficient household income 
to enable children to develop the skills they need to succeed in later life. Policies that focus 
on education, while important and welcome, are not sufficient. The most effective are likely 
to be focused on increasing equity (European Commission/EACEA/Eurydice, 2020) aiming 
to level up by addressing the causes (such as low income, housing costs, limited access to 
good jobs (JRF, 2020)) rather than the effects. Then policies can focus holistically on early 
intervention to address the consequences of disadvantage (across joined up policies 
involving health, housing, social care, youth services etc. as well as education) and work 
on area-based strategies to address local/community-based issues to reduce absolute and 
relative poverty. 


It will be especially important to ensure that there is a system of support around low- 
income families as the COVID-19 crisis continues, making sure that basic needs are met 
in the immediate term. This includes ensuring access to school, especially for young 
people from disadvantaged areas who do not have study space or access to digital 
resources at home (Sharp et a/., 2020). 


5.2 Recommendations 


Programmes that support metacognition could be beneficial for 
disadvantaged pupils 


Although causality cannot be inferred from this analysis alone, there is a wealth of 
evidence and support suggesting that teaching and using metacognitive strategies can 
help to raise the attainment of disadvantaged pupils (EEF, 2018; Macleod et a/., 2015; 
Welsh Government, 2015). This is supported by the findings that in all three countries, 
where the use of metacognitive strategies was associated with resilience, in that pupils 
who were disadvantaged, but high-achieving, were more likely to use these strategies than 
low-achieving, disadvantaged pupils. Metacognitive strategies can be, and arguably 
should be, taught in conjunction with subject specific content that will help to cement these 
as transferable skills (EEF, 2018). 
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Work towards countering the belief that intelligence is fixed 


Resilient pupils were more likely to have a growth mind-set, that is, they saw intelligence 
as something that can be changed. It may be that having this mind-set would encourage 
pupils to work harder at subjects in which they are not yet achieving or that pupils who 
work hard learn that your intelligence can grow. Either way, fostering the concept of a 
growth mind-set for disadvantaged pupils would encourage them to overcome barriers. 


Investigate the connection between aspiration and resilience to identify 
how they are related 


The findings from this analysis indicates an association between high aspiration and 
resilience, however it does not determine whether high aspiration is a cause or effect of 
resilience. Further exploration would help to determine how and where targeted support 
would be most beneficial for disadvantaged pupils. Given the effect of COVID-19 on 
economies around the world, it is important that disadvantaged pupils believe that their 
hard work at school could lead to rewarding opportunities in the future. 
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Appendix A 
Variables included in the profiling 


All derived variables from the Pupil Questionnaire were included in the profiling analyses. 
Chapter 16 of the PISA 2018 Technical Report has more details of the derived variables. 


A number of single questions were included in the profiling stage based on previous 
research): 


e In the last two full weeks of school, how often: | <skipped> a whole school day. 
e About how much time do you usually spend reading for enjoyment? 
e Agree: Your intelligence is something about you that you can't change very much. 


The single question ‘Overall, how satisfied are you with your life as a whole these days?’ 
was also included at this stage to complement the simultaneous analysis on wellbeing 
using PISA 2018 data (Kuhn et a/., 2021). 


Variables included in the multi-level logistic regression model 


Table 4 includes the variables from national data (NPD or Northern Ireland School Census 
data) included in the respective country’s regression. 
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Table 4 National variables included in each multi-level logistic regression 


Variable Description 
E EVER1 England FSM 6 
E KS2_1 England KS2 APS 
Northern Ireland pupil entitled to free school 
any year in past six years 
N_FSM_1 
Northern Ireland religion of the pupil 
N_RELI1 
Northern Ireland Religion by gender 
Note: Interaction between religion and 
RXS gender added to the Northern Ireland 
regression 
Northern Ireland school type — Non- 
grammar 
Note: Grammar was included as the 
SCHTYPE1 reference category 
Northern Ireland school type — Independent 
Note: Grammar was included as the 
SCHTYPE2 reference category 
LANGTE1 Language of test 
W_NPD_1 Wales- Study Welsh 
pupil entitled to free school any year in past 
W_NPD_3 six years 


Table 5 includes the PISA variables included in each country’s multi-level logistic 
regression. These were selected for the following reasons: 


1; The variables correlated highly (above 0.15) at the Profiling Stage (for example, 
Perception of difficulty of the PISA test; Enjoy/Like reading) 


2. They acted as a control variable (for example, age; gender) 
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Previous research suggested them as an area of interest (for example, In the last 


two full weeks of school, how often: | <skipped> a whole school day; emotional 
support from parents) 


4. To explore the relationship between socio-economic status and wellbeing (for 
example, life-satisfaction and positive emotions 
Table 5 PISA variables included in each multi-level logistic regression 
Variable Description 
AGE Age 
ATTLNACT Attitude towards school: learning activities (WLE) 
BSMJ Students expected occupational status (SEI) 
School changes (Only available for England as variable derived from 
CHANGE Educational Careers Questionnaire) 
DISCLIMA Disciplinary climate in test language lessons (WLE) 
EFFORT1 How much effort did you put into this test? 
EFFORT2 How much effort would you have invested? 
EMOSUPS Emotional support from parents 
EUDMO Eudemonia: meaning in life (WLE) 
GFOFAIL General fear of failure (WLE) 
HEDRES Home educational resources (WLE) 
HISE| Index highest parental occupational status 
IMMI1 Second generation immigrant 
IMMI2 First generation immigrant 
JOYREAD Enjoy/Like reading (WLE) 
| | Language spoken with their school mates for students who do not —s| 
LANGSC1 speak the test language at home 
METASPAM Meta-cognition: assess credibility 
METASUM Meta-cognition: summarising 
PERCOMP Perception of competitiveness at school (WLE) 
PISADIFF Perception of difficulty of the PISA test (WLE) 
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Variable Description 

RESILI1 Resilience 

S$C012Q1 Student admission to school: Residence in a particular area 
Student admission to school: Parents' endorsement of the instructional 

$C012Q2 or religious philosophy of the school 

$C012Q3 Student admission to school: Recommendation of feeder schools 
Student admission to school: Whether the student requires or is 

S$C012Q4 interested in a special programme 

SCREAD1 Self-concept of reading: Perception of difficulty (WLE) 

SCREAD2 Self-concept of reading: Perception of competence (WLE) 

ST004D1 Student (Standardized) Gender 

ST016Q1 Life-satisfaction 
In the last two full weeks of school, how often: | <skipped> a whole 

ST062Q1 school day. 

ST175Q1 About how much time do you usually spend reading for enjoyment? 
Agree: Your intelligence is something about you that you can't change 

$T184Q1 very much. 

ST225ALL Highest level of studies selected 

SWBP Positive emotions 

UNDREM Meta-cognition: understanding and remembering 

WORKMAST Work mastery (WLE) 
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